Communication over the Internet begins with a request from a client browser for information located at a specified domain. For example, an Internet user on a client computer may type “www.acme-co.com/products” in a web browser, which is a request for the information located at an address corresponding to www.acme-co.com. Such a request is known as a DNS request, and is routed to name servers (DNS servers) for translation into the Internet Protocol (IP) address corresponding to the domain. For example, the name server may translate www.acme-co.com into the IP address 122.304.22.1. After translating the request into an IP address, the name server provides the address to the client and the client may retrieve the desired information.
Because of the volume of DNS packet requests that name servers must process each day, many different and geographically diverse name servers exist to handle the DNS packet requests. Each name server, also known as an edge site, may have a network recorder to passively analyze and record the network traffic and log this information into files written to a storage medium. One example of this recorded information is a packet capture (PCAP) file. The PCAP files recorded at a name server location are generally kept at the location. Because the PCAP files are large, not indexed, and geographically isolated, the data recorded in the PCAP files is not easily accessible for analysis.
In addition to DNS packet requests, a network recorder may capture other types of network traffic, such as NetFlow™ records and HTTP, POP, and SMTP requests. This information may also be stored in PCAP files or some other type of log file. Like the information in the DNS logs, this information may also be geographically isolated and not easily accessible for analysis.
Therefore, it is desirable to introduce tools to allow name server owners to collect and process this information in a central repository so that trends can be analyzed and statistical models created.